1. Field of the Invention
This invention relates to the field of warped video image processing. Specifically, embodiments of this invention include methods, apparatus, and computer program products for specifying objects in a warped video image and tracking the specified objects as they move through the video over time.
2. Background
A warped video image includes a set of video frames each frame containing non-rectilinear data. Such a warped video image can be created by capturing an image through a wide-angle lens, a catadioptric lens or some other distorting lens. The warped video image can be sent to a user/viewer who can then select portions of the data for transformation into a rectilinear form for presentation (such as by display).
Most video cameras only record a view within a small viewing angle. Thus, a typical video camera only captures an image in the direction that the camera is aimed. Such conventional cameras force viewers to look only at what the camera operator chooses to focus upon.
A video camera equipped with a wide-angle lens captures warped video images. For example, a panoramic camera constructed with a catadioptric lens captures a warped annular image that represents a substantially 360-degree scene that extends through a horizon line. Other wide-angle lenses (for example, a fish-eye lens) can generate warped circular images and may capture substantially a hemisphere (180-degree) view.
A portion of the warped video image can be transformed to provide a rectilinear representation of that portion of the warped video image. Thus, a viewer/user of the warped video image can individually select what portion of the warped video image is to be presented. As the user/viewer focuses on a moving object, the user/viewer must constantly adjust the presented view to track the moving object as it moves through the warped video image.
In some circumstances (for example, in a television broadcast control room), a user selects a view for presentation to other viewers. In this case, the viewer is unable to independently select the view because the user has done so.
There are many known techniques for tracking moving objects in a video. Some of these are described by U.S. Pat. No. 5,548,659, by Okamoto and entitled Method and Apparatus for Detecting Changes in Dynamic Images; by U.S. Pat. No. 5,877,804, by Otsuki et al. and entitled Method and Apparatus for Moving Object Detection; by U.S. Pat. No. 5,537,155, by O'Connell et al. and entitled Method for estimating Motion in a Video Sequence; U.S. Pat. No. 6,072,494, by Nguyen and entitled Method and Apparatus for Real-Time Gesture Recognition; by U.S. Pat. No. 5,434,617, and by Bianchi and entitled Automatic Tracking Camera Control System. In addition, other teachings are from Statistical Background Modeling for Tracking With a Virtual Camera, by Rowe and Blake; from Divide and Conquer: Using Approximate World Models to Control View-Based Algorithms, by Bobick and Pinhanez, and from Approximate World Models: Incorporating Qualitative and Linguistic Information into Vision Systems, by Pinhanez and Bobick. However, these techniques are not known to have been used to track movement or objects through a warped video image, nor to other aspects of the inventions disclosed within.
Conventional images are often delivered by electronic means. For example, television and the Internet deliver conventional images across wired and wireless electronic media. However, there are no standard means of delivering real-time panoramic images or wide-angle images electronically. Since panoramic images are so large and include so much data, it is difficult to deliver these images using conventional image transmission techniques. To further compound the problem, real-time motion panoramic images require a very high bandwidth channel for electronic distribution.
It would be advantageous to provide methods, apparatus, systems and program product solutions that allow a user/viewer to select an object-of-interest and to track it in real-time as the object-of-interest moves through a warped video image. In addition it would be advantageous to allocate bandwidth to the portions of the warped video image that include the tracked object-of-interest. Furthermore, it would be advantageous to monitor when the object-of-interest moves into an area-of-interest and to respond to such an occurrence.